Skip to content

Merge tag darwin/darwin-008 on darwin/trunk. #252

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 50 commits into from
May 25, 2017
Merged

Conversation

das
Copy link
Contributor

@das das commented May 25, 2017

This includes the github PR's that we've pulled back into the Apple internal repository starting with the merge of PR #216 up to the merge of PR #209, along with any changes made to them during internal review and testing on Darwin:

#216 from dgrove-oss/das-darwin-libdispatch-806-merge-master-linux-fixes
#218 from apple/das-api-availability-build-fix
#221 from apple/das-disable-linux-membarrier-include
#223 from apple/eng/PR-SR-4039
#226 from dgrove-oss/fix-compiler-warnings
#227 from ktopley-apple
#230 from ktopley-apple
#232 from ktopley-apple
#234 from linux-on-ibm-z/fix_compile_error
#236 from bcardosolopes/master
#238 from ktopley-apple
#196 from compnerd/cmake
#233 from amboar/master
#243 from compnerd/swiftDispatch
#231 from dgrove-oss/SR-4201-signalsource
#244 from compnerd/ptrsize
#245 from compnerd/windows-cleanups
#246 from compnerd/build-dependencies
#206 from dgrove-oss/internal-pwq-impl
#209 from weissi/jw-fix-libdispatch.a

das and others added 30 commits May 24, 2017 20:13
Signed-off-by: Daniel A. Steffen <dsteffen@apple.com>
Signed-off-by: Daniel A. Steffen <dsteffen@apple.com>
Signed-off-by: Daniel A. Steffen <dsteffen@apple.com>
Signed-off-by: Daniel A. Steffen <dsteffen@apple.com>
Signed-off-by: Daniel A. Steffen <dsteffen@apple.com>
The static version (`libdispatch.a`) that can be compiled using
`./configure --enable static=yes` was missing all the Swift overlay symbols
(everything in `swift_overlay.o`). The reason for that is that the linker is
invoked through libtool which wants `.lo` files but the Swift overlay got passed
as a `.o` file. This first of all leads to this warning:

    *** Warning: Linking the shared library libdispatch.la against the non-libtool
    *** objects [...]/swift_overlay.o is not portable!

And the result is that libtool doesn't include `swift_overlay.o` when putting
together the static library `libdispatch.a`.

This patch fixes this problem is a pretty crude way. The real problem is that
libtool doesn't understand Swift. So it can't be used when compiling
`swift_overlay.o`. In order to still get a `.lo` file this patch tricks libtool
into generating one from the `swift_overlay.o` generated by `swiftc`. It's very
hacky but I'm not sure what else to do.

Signed-off-by: Daniel A. Steffen <dsteffen@apple.com>
fix libdispatch.a to include Swift overlay symbols

Signed-off-by: Daniel A. Steffen <dsteffen@apple.com>
Collection of small fixes to dispatch-806 merge to
allow code to compile/link/run on Linux.

Signed-off-by: Daniel A. Steffen <dsteffen@apple.com>
…rge-master-linux-fixes

Linux fixes for dispatch-806 merge

Signed-off-by: Daniel A. Steffen <dsteffen@apple.com>
Signed-off-by: Daniel A. Steffen <dsteffen@apple.com>
provide fallback definitions for all API_* macros

Signed-off-by: Daniel A. Steffen <dsteffen@apple.com>
add dependency on linux-libc-dev to INSTALL.md for other architectures

Signed-off-by: Daniel A. Steffen <dsteffen@apple.com>
remove unused dependency on linux/membarrier.h for intel to fix CI build

Signed-off-by: Daniel A. Steffen <dsteffen@apple.com>
Also reorder all vtable entries to be in the same order (push, invoke,
wakeup).

Reenable dispatch_context_for_key.

SR: SR-4039

Signed-off-by: Daniel A. Steffen <dsteffen@apple.com>
queue_specific_queue is missing its do_push vtable entry

Signed-off-by: Daniel A. Steffen <dsteffen@apple.com>
Collection of small fixes to eliminate compiler
warnings when building libdispatch and its tests.

Signed-off-by: Daniel A. Steffen <dsteffen@apple.com>
fixes for compiler warnings

Signed-off-by: Daniel A. Steffen <dsteffen@apple.com>
(Radar rdar://problem/30518517)

Signed-off-by: Daniel A. Steffen <dsteffen@apple.com>
(Radar 30699743)

Signed-off-by: Daniel A. Steffen <dsteffen@apple.com>
Signed-off-by: Daniel A. Steffen <dsteffen@apple.com>
…ting the fact that a relative or non-existent path is invalid.

(Radar 31115994)

Signed-off-by: Daniel A. Steffen <dsteffen@apple.com>
… model and does not require membarrier.

Signed-off-by: Daniel A. Steffen <dsteffen@apple.com>
Fix compile error on s390x in shims/lock.h

Signed-off-by: Daniel A. Steffen <dsteffen@apple.com>
libdispatch has an umbrella header, dispatch.h, that should be used for
including all of the other headers.  This is enforced via
__DISPATCH_INDIRECT__.

Since it isn't legal to include one of the other headers on their own,
it isn't logical to have submodules for them.  Moreover, submodules
*should* have local visibility (be unaware of the context they're
included from); meaning that __DISPATCH_INDIRECT__ will never be
defined.

Logically, all of the headers are part of a single module/interface.

Patch by me & Duncan P. Exon Smith

Signed-off-by: Daniel A. Steffen <dsteffen@apple.com>
libdispatch: Avoid submodules for headers that should not be included

Signed-off-by: Daniel A. Steffen <dsteffen@apple.com>
(Radar 31585625)

Signed-off-by: Daniel A. Steffen <dsteffen@apple.com>
This is far from complete, but is sufficient to build a Linux version of
libdispatch.  It shows what a potential cmake based build system could
look like, and if desired can be completed to build all the various
flavours with cmake.

Signed-off-by: Daniel A. Steffen <dsteffen@apple.com>
build: add a cmake based build system

Signed-off-by: Daniel A. Steffen <dsteffen@apple.com>
Building swift-corelibs-libdispatch on powerpc64le under Linux lead to
the following build failure:

	/bin/bash ../libtool  --tag=CXX   --mode=compile /home/ubuntu/swift-dev/build/buildbot_incremental/llvm-linux-powerpc64le/bin/clang++ -DHAVE_CONFIG_H -I. -I../config  -I.. -I.. -I../private -DDISPATCH_USE_DTRACE=0 -I../libpwq/include -Wall -fvisibility=hidden -momit-leaf-frame-pointer  -isystem /usr/include/bsd -DLIBBSD_OVERLAY  -fblocks -I../src/BlocksRuntime -std=gnu++11 -fno-exceptions -O2 -c -o libdispatch_la-block.lo `test -f 'block.cpp' || echo './'`block.cpp
	libtool: compile:  /home/ubuntu/swift-dev/build/buildbot_incremental/llvm-linux-powerpc64le/bin/clang++ -DHAVE_CONFIG_H -I. -I../config -I.. -I.. -I../private -DDISPATCH_USE_DTRACE=0 -I../libpwq/include -Wall -fvisibility=hidden -momit-leaf-frame-pointer -isystem /usr/include/bsd -DLIBBSD_OVERLAY -fblocks -I../src/BlocksRuntime -std=gnu++11 -fno-exce
	ptions -O2 -c block.cpp  -fPIC -DPIC -o .libs/libdispatch_la-block.o
	In file included from block.cpp:32:
	In file included from ./internal.h:628:
	In file included from ./shims.h:171:
	./shims/lock.h:550:3: error: use of undeclared identifier 'DISPATCH_INTERNAL_CRASH'
			DISPATCH_INTERNAL_CRASH(errno, "sys_membarrier not supported");
			^
	1 error generated.
	Makefile:701: recipe for target 'libdispatch_la-block.lo' failed
	make[2]: *** [libdispatch_la-block.lo] Error 1
	make[2]: Leaving directory '/home/ubuntu/swift-dev/swift-corelibs-libdispatch/src'
	Makefile:541: recipe for target 'all' failed
	make[1]: *** [all] Error 2
	make[1]: Leaving directory '/home/ubuntu/swift-dev/swift-corelibs-libdispatch/src'
	Makefile:457: recipe for target 'all-recursive' failed
	make: *** [all-recursive] Error 1

Include ordering in internal.h is tightly constrained, so open-code the macro
to avoid the dependency problem.

Signed-off-by: Andrew Jeffery <andrew@aj.id.au>
Signed-off-by: Daniel A. Steffen <dsteffen@apple.com>
lock: Avoid use of undefined DISPATCH_INTERNAL_CRASH

Signed-off-by: Daniel A. Steffen <dsteffen@apple.com>
compnerd and others added 19 commits May 24, 2017 20:13
Introduce support for building the swift SDK overlay into the CMake
based build system.  This was the one piece which was missing from the
original cmake port of the build system.

Signed-off-by: Daniel A. Steffen <dsteffen@apple.com>
build: attempt to add swift support to cmake

Signed-off-by: Daniel A. Steffen <dsteffen@apple.com>
The observed bug is that signals sent to the main thread
after it called dispatch_main were not triggering the
event handler of a DispatchSourceSignal that had been
previously created and activated to handle that signal.

This patch addresses the problem by:

- export _dispatch_sigmask() as the way to block signals in a process
  using dispatch
- call it when dispatch_main() is called
- each time dispatch registers a signal handler, register a sigaction that
  will catch signals for threads with wrong masks and will fix the thread
  configuration then raise the signal again by directing a pthread_kill
  to the manager thread so that it is delivered to the signalfd as expected.

Patch by Pierre Habouzit <phabouzit@apple.com> from PR-231
with minor compilation fixes by Dave Grove.

Signed-off-by: Daniel A. Steffen <dsteffen@apple.com>
SR-4201: DispatchSourceSignal not working on Linux

Signed-off-by: Daniel A. Steffen <dsteffen@apple.com>
Signed-off-by: Daniel A. Steffen <dsteffen@apple.com>
This introduces a new macro, `DISPATCH_PTR_SIZE` which is defined much
like `LLVM_PTR_SIZE`.  We can gracefully fallback to alternate means of
checking the width of the pointer and use this rather than `__LP64__` to
determine if we are on a 64-bit host.

Signed-off-by: Daniel A. Steffen <dsteffen@apple.com>
Create a DISPATCH_PTR_SIZE macro and use it

Signed-off-by: Daniel A. Steffen <dsteffen@apple.com>
sys/cdefs.h and unistd.h are not always available.  We already have
checks for these, guard the inclusion appropriately.

Signed-off-by: Daniel A. Steffen <dsteffen@apple.com>
Rather than checking for non-Linux and including the target specific
header, invert the check and include the linux target header if
`__linux__` is defined.  This makes it easier to target other platforms.

Signed-off-by: Daniel A. Steffen <dsteffen@apple.com>
Address some of the portability considerations for the build flags.
This improves some of the conditions for building for Windows.

Signed-off-by: Daniel A. Steffen <dsteffen@apple.com>
Windows cleanups

Signed-off-by: Daniel A. Steffen <dsteffen@apple.com>
Signed-off-by: Daniel A. Steffen <dsteffen@apple.com>
This fixes a typo in the build system which is important because it
fixes the dependency tracking and the output location for the swift doc
that is generated for the swift SDK overlay.

Signed-off-by: Daniel A. Steffen <dsteffen@apple.com>
build: fix typo in build system

Signed-off-by: Daniel A. Steffen <dsteffen@apple.com>
Provide an implementation of the pthread_workqueue
functionality (libpwq) as a fully integrated component of
libdispatch.  Integration of the workqueue implementation
into the libdispatch code base simplifies the process of
evolving the APIs between the two layers and thus
prepares for future optimization and enhancements.

The overall design is to augment libdispatch's existing
pthread_root_queue option with periodic user-space monitoring
of the status of a root queue's worker threads and to
oversubscribe the workqueue when not enough workers are
observed to be runnable when the queue has available work.

Initially, the integrated pthread_workqueue is only
enabled by default on Linux. The current monitoring
implementation relies on Linux-specific code to dynamically
estimate the number of runnable worker threads by reading /proc.
Porting the monitoring code to non-Linux platforms would entail
providing similar functionality for those platforms.

Remove libpwq git submodule and autotools support for
building libpwq from source as part of building libdispatch.

Signed-off-by: Daniel A. Steffen <dsteffen@apple.com>
implement pthread_workqueue within libdispatch

Signed-off-by: Daniel A. Steffen <dsteffen@apple.com>
Signed-off-by: Daniel A. Steffen <dsteffen@apple.com>
Signed-off-by: Daniel A. Steffen <dsteffen@apple.com>
@das
Copy link
Contributor Author

das commented May 25, 2017

@swift-ci please test

@das das requested a review from dgrove-oss May 25, 2017 03:40
@das
Copy link
Contributor Author

das commented May 25, 2017

@swift-ci please test

1 similar comment
@das
Copy link
Contributor Author

das commented May 25, 2017

@swift-ci please test

@das das self-assigned this May 25, 2017
Copy link
Contributor

@dgrove-oss dgrove-oss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks fine to me.

src/shims.h Outdated
@@ -41,6 +41,8 @@
#if HAVE_PTHREAD_WORKQUEUES
#if __has_include(<pthread/workqueue_private.h>)
#include <pthread/workqueue_private.h>
#elif DISPATCH_USE_INTERNAL_WORKQUEUE
#include "event/workqueue_internal.h"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think since you merged #248, the elif is dead code. But fine to merge as is to save time; I can double check after the merge and submit a PR to remove this if I'm right.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

duh, I had resolved the merge conflict the wrong way around, thanks for spotting

squashed the fix

Signed-off-by: Daniel A. Steffen <dsteffen@apple.com>
@das das force-pushed the das-darwin-008-merge-master branch from 7406bad to 81a67c9 Compare May 25, 2017 12:22
@das
Copy link
Contributor Author

das commented May 25, 2017

@swift-ci please test

@das das merged commit acb1b1e into master May 25, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants